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121. Rong Zhang and Alexander I. Rudnicky, Word Level Confidence Annotation using 
Combinations of Features, Proceedings of Eurospeech 2001 (Aalborg, Denmark).. 


122. Chotimongkol, A. and Rudnicky, A.I. N-best Speech Hypotheses Reordering Using Linear 
Regression Proceedings of Eurospeech 2001 (Aalborg, Denmark), pages 1829-1832. 


123. Zhang, R. and Rudnicky, A.I. Word Level Confidence Annotation using Combinations of 
Features Proceedings of Eurospeech 2001 (Aalborg, Denmark), pages 2105-2108. 


124. Carpenter, P., Jin, C., Wilson, D., Zhang, R., Bohus, D., Rudnicky, A. Is this Conversation 
on Track? Proceedings of Eurospeech 2001 (Aalborg, Denmark), pages 2121-2124. 


125. Du Bois, T. and Rudnicky, A.I. An Open Concept Metric for Assessing Dialog System 
Complexity Proceedings of the 2001 Automatic Speech Recognition and Understanding 
Workshop (Madonna di Campiglio, Italy) Paper a01td123. 


126. Bohus, D. and Rudnicky, A.I. Modeling the Cost of Misunderstanding Errors in the CMU 
Communicator Proceedings of the 2001 Automatic Speech Recognition and Understanding 
Workshop (Madonna di Campiglio, Italy) Paper a01db097. 


127. Oh, A. H. and Rudnicky, A. Stochastic language generation for spoken dialogue systems. 
ANLP/NAACL 2000 Workshop on Conversational Systems, May 2000, pp. 27-32. 


128. Xu, W. and Rudnicky, A. Task-based dialog management using an agenda. ANLP/NAACL 
2000 Workshop on Conversational Systems, May 2000, pp. 42-47. 


129. Rudnicky, A., Bennett, C., Black, A., Chotimongkol, A., Lenzo, K., Oh, A., Singh, R. Task 
and domain specific modeling in the Carnegie Mellon Communicator system Proceedings of 
ICSLP, 2000 (Beijing, China) Paper G4-01. 


130. Xu, W. & Rudnicky, A. Can artificial neural networks learn language models? Proceedings 
of ICSLP 2000 (Beijing, China). Paper M1-13. 


131. Xu, W. and Rudnicky, A. Language modeling for dialog system? Proceedings of ICSLP 
2000 (Beijing, China). Paper B1-06. 


132. Frederking, R., Hogan, C, and Rudnicky, A. A New Approach to the Translating Telephone. 
In Proceedings of the Machine Translation Summit VI: MT in the Great Translation Era, 
Singapore, September 1999. 


133. Rudnicky, A. and Xu W. An agenda-based dialog management architecture for spoken 
language systems. IEEE ASRU Workshop, December 1999, p. I-337. 


134. Eskenazi, M., Rudnicky, A., Gregory, K., Constantinides, P., Brennan, R., Bennett, C., 
Allen, J. Data Collection and Processing in the Carnegie Mellon Communicator. Proceedings 
of Eurospeech, 1999, 6, 2695-2698. 


135. Rudnicky, A., Thayer, E., Constantinides, P., Tchou, C., Shern, R., Lenzo, K., Xu W., Oh, 
A. Creating natural dialogs in the Carnegie Mellon Communicator system. Proceedings of 
Eurospeech, 1999, 4, 1531-1534. 


136. Constantinides, P. and Rudnicky, A. Dialog analysis in the Carnegie Mellon Communicator. 
Proceedings of Eurospeech, 1999, 1, 243-246. 


137. Constantinides, P., Hansma, S., Tchou, C. and Rudnicky, A. A schema-based approach to 
dialog control. Proceedings of ICSLP. 1998, Paper 637. 


138. Frederking, R., Rudnicky, A. and C. Hogan DIPLOMAT: a voice-to-voice translation 
system. ACL Workshop on Spoken Language Translation, Madrid, 1997, pp. 61-66. 


139. Rudnicky, A.I., Reed, S. and Thayer, E.H. SpeechWear: A mobile speech system, 
Proceedings of ICSLP, 1996. 
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140. Rudnicky, A.I. Hub 4: Business Broadcast News. Proceedings of the 1996 ARPA Workshop 
on Speech Recognition Technology, 1996, pp.8-11. 


141. Hauptmann, A.G., Witbrock, M.J., Rudnicky, A.I., and Reed, S. Speech for Multimedia 
Information Retrieval. UIST-95 Proceedings of the User Interface Software Technology 
Conference, Pittsburgh, November 1995. 


142. Rudnicky, A.I. Language modeling with limited domain data, Proceedings of the Arpa 
Spoken Language Systems Technology Workshop. San Mateo: Morgan Kaufmann, 1995, pp. 
66-69. 


143. Rudnicky, A.I. Factors affecting choice of speech over keyboard and mouse in a simple data- 
retrieval task. Proceedings of EUROSPEECH'93, 1993, 2161-2164. 


144. L. Hirschman, M. Bates, D. Dahl, W. Fisher, J. Garofolo, D. Pallett, K. Hunicke-Smith, P. 
Price, A. Rudnicky, E. Tzoukermann. Multi-site data collection and evaluation in spoken 
language understanding. Proceedings of the Arpa Workshop on Human Language 
Technology. San Mateo: Morgan Kaufmann, 1993, 19-24. 


145. Rudnicky, A.I. Mode preference in a simple data-retrieval task. Proceedings of the Arpa 
Workshop on Human Language Technology. San Mateo: Morgan Kaufmann, 1993, 364-369. 


146. Teal, S.L. and Rudnicky, A.I. A performance model of system delay and user strategy. 
Proceedings of the CHI conference, 1992, 295-306. 


147. Rudnicky, A.I., Lunati, J.-M. and Franz, A.M. Spoken language recognition in an office 
management domain. Proceedings of the Intl. Conf. on Acoustics, Speech, and Signal 
Processing, 1991, 829-832. 


148. Lunati, J.-M. and Rudnicky, A.I. Spoken language interfaces: The OM system. 
Proceedings of the CHI conference, April 1991, 453-454. 


149. Rudnicky, A.I. and Hauptmann, A.G. Models for evaluating interaction protocols in speech 
recognition. Proceedings of the CHI conference, April 1991, 285-291. 


150. Rudnicky, A.I. System response delay and user strategy selection. Invited poster, CHI 
Conference, 1990. 


151. Rudnicky, A.I., Sakamoto, M.H. and Polifroni, J.H. Spoken language interaction in a 
spreadsheet task. In D. Diaper et al., Human-Computer Interaction -- INTERACT'90, New 
York: Elsevier, 1990, 767-772. 


152. A.G. Hauptmann and A.I. Rudnicky, Speaking, Typing and Gesturing. In CHI-90 Workshop 
on Multimedia and Multimodal Interfaces, International Conference on Human Factors in 
Computing Systems, Seattle WA, April 1990. 


153. Rudnicky, A.I., Sakamoto, M.H., and Polifroni, J.H. Spoken language interaction in a goal- 
directed task. Proceedings of the Intl. Conf. on Acoustics, Speech, and Signal Processing, 
1990, 1, 45-48. 


154. Lunati, J.-M. and Rudnicky, A.I. The design of a spoken language interface, Proceedings of 
the Third DARPA Speech and Natural Language Workshop, San Mateo: Morgan-Kaufmann, 
1990, p. 225-229. 


155. Hauptmann, A.G. and Rudnicky, A.I. A comparison of speech versus typed input. 
Proceedings of the Third DARPA Speech and Natural Language Workshop, San Mateo: 
Morgan-Kaufmann, 1990, p. 219-224. 
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156. Rudnicky, A.I., Sakamoto, M.H., and Polifroni, J.H. Evaluating spoken language interaction. 
Proceedings of the DARPA Workshop on Speech and Natural Language, San Mateo: Morgan 
Kaufmann, October 1989, 150-159. 


157. Rudnicky, A.I. The design of voice-driven interfaces. Proceedings of the DARPA 
Workshop on Speech and Natural Language, San Mateo: Morgan Kaufmann, February 1989, 
120-124. 


158. Rudnicky, A.I., Li, Z., Polifroni, J.H., Thayer, E.H, and Gale, J. An unanchored matching 
algorithm for lexical access. Proceedings of the Intl. Conf. on Acoustics, Speech, and Signal 
Processing, 1988, 1, 469-472. 


159. Murveit, H., Weintraub, M., Cohen, M., Bernstein, J., and Rudnicky, A.I. Three approaches 
for the design of a lexical access module for the DARPA Angel speech recognition system. 
Proceedings of the Intl. Conf. on Acoustics, Speech, and Signal Processing, 1987, 837-840. 


160. Rudnicky, A.I., Baumeister, L.K., DeGraaf, K.H., and Lehmann, E. The lexical access 


component of the CMU continuous speech recognition system. Proceedings of the Intl. Conf. 
on Acoustics, Speech, and Signal Processing, 1987, 376-379. 


d. Other Refereed Publications 


161. Lee, K-F, Hauptmann, A.G. and Rudnicky, A.I. The spoken word. Byte, 1990, 15(7), 225- 
232. 


162. Rudnicky, A.I. and Hauptmann, A.G. Errors, Repetition, and Contrastive Emphasis in 
Speech Recognition. AAAI Symposium on Spoken Language Systems, March 1989. 


163. Rudnicky, A.I. and Stern, R.M. Spoken language research at Carnegie Mellon. Speech 
Technology, 1989, 4(4), 38-43. 


164. Rudnicky, A.I. Goal-directed speech in a spoken language system, Journal of the Acoustical 
Society of America, 1989, 86, S76(A). 


165. Polifroni, J.H. and Rudnicky, A. I. Modeling lexical stress in read and spontaneous speech. 
Journal of the Acoustical Society of America, Journal of the Acoustical Society of America, 
1989, 86, S77(A). 


166. Rudnicky, A.I., Brennan, R.A, Polifroni, J.H., Thayer, E.H. Interactive problem solving with 
speech. Journal of the Acoustical Society of America, 1988, 84, S213(A). 


167. Rudnicky, A.I. and Li, Z. Prosodic information in lexical access. Paper presented at the 
DARPA Speech Workshop, June 1988. 


168. Rudnicky, A.I. Using features to empirically generate pronunciation networks. Journal of 
the Acoustical Society of America, 1987, 82, (A). 


169. Rudnicky, A.I. Speaker-independent recognition of vocalic segments. Journal of the 
Acoustical Society of America, 1984, 61, S46(A). 


170. Rudnicky, A.I., Waibel, A.H., and Krishnan, N.V. Using zero crossing counts to provide 
discriminative information in isolated word recognition. Journal of the Acoustical Society of 
America, 1981, 70, S60(A). 


171. Rudnicky, A.I. Units of perception in phoneme monitoring. Paper presented at the 21st 
meeting of the Psychonomic Society, St. Louis, November, 1980. 
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172. Rudnicky, A.I. The role of structure in speech perception: Evidence from phoneme 
monitoring. Paper presented at the Spring Meeting of the Midwestern Psychological 
Association, St. Louis, May, 1980. 


173. Rudnicky, A.I. The perception of speech in an unfamiliar language. In Speech 
Communications papers presented at the 97th meeting of the Acoustical Society of America, 
J. Wolf and D. Klatt (Eds.) New York: Acoustical Society of America, 1979. 


174. Rudnicky, A.I. and Cole, R.A. Vowel identification and subsequent context. Journal of the 
Acoustical Society of America, 1977, 61, S39(A). 


175. Rudnicky, A.I. and Cole, R.A. Selective adaptation produced by ongoing speech. Journal of 
the Acoustical Society of America, 1976, 59, S26(A). 


e. UnRefereed Conference/Workshop Papers 


176. Rudnicky, A. I. and Oh, A. H. Dialog Annotation for Stochastic Generation. Proceedings of 
the ISLE Workshop on Dialogue Tagging for Human Computer Interaction, Edinburgh, U.K., 
December 15-17th, 2002. [invited paper] 


177. Lunati, J-M and Rudnicky, A.I. Human factors in the design of a spoken language system. 
Proceedings of Speech Tech'91, April 1991. 


f. Technical Reports 


178. Bohus, D. and Rudnicky, A. Integrating multiple knowledge sources for utterance-level 
confidence annotation in the CMU Communicator spoken dialog system. Carnegie Mellon 
University School of Computer Science Technical Report CMU-CS-02-190, November 2002. 


179. Miller, B.W, Hwang C.H, Lee Y., Roberts J., Rudnicky A.I, "The I3S Project: A Mixed, 
Behavioral and Semantic Approach to Discourse/Dialogue Systems." MCC Technical Report 
13S-107-00, January 2000. 


180. Rudnicky, A.I The design of spoken language interfaces. Carnegie Mellon University 
School of Computer Science Technical Report CMU-CS-90-118, March 1990. 


181. Rudnicky, A. I. and Hauptmann, A. G. Conversational interaction with speech systems. 
Carnegie Mellon University School of Computer Science. Technical Report CMU-CS-89- 
203, December 1989. 


182. Rudnicky, A.I. and Sakamoto, M.H. Transcription conventions for spoken language 
research. Carnegie Mellon University School of Computer Science Technical Report CMU- 
CS-89-194, November 1989. 


183. Rudnicky, A.I. Waibel, A.H., and Krishnan, N. Adding a zero-crossing count to spectral 
information in template-based speech recognition. Carnegie-Mellon Computer Science 
Department, Technical Report CMU-CS-82-140, October, 1982. 


g. Unpublished Papers 


184. Ayoob, E., Bodine, K., Bohus, D., Rudnicky, A. I. & Siegel, J. Users’ Performance and 
Preferences for Online Graphic, Text and Auditory Presentation of Instructions. Carnegie 
Mellon University, 2004. 


185. Damiba, B. A. & Rudnicky, A. I. Internationalizing Speech Technology through Language 
Independent Lexical Acquisition. Carnegie Mellon University, 1998. 
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186. Damiba, B. A. & Rudnicky, A. I. Language-Independent Lexical Acquisition. Carnegie 
Mellon University, 1997. 


187. Rudnicky, A.I. Speech Interface Guidelines. Carnegie Mellon University, May, 1996. 


h. Software Artifacts 


188. TeamTalk, 2005-2009. A spoken dialog system supporting multiple participants (with 
Harris, Pappu, Li, Frisch and others). 


189. Logios, 2007-2009. A set of tools for creating speech system knowledge bases (with Harris) 


190. LMTool, 1996-2009. A set of web-based tools for creating language models and 
pronunciation dictionaries for the Sphinx ASR system. 


191. Carnegie Mellon Communicator, 1998-2002. A telephone-based spoken language travel 
planning system. (with Xu, Oh, Constantinides, Thayer and others). 


192. Speech Workbench, 1997-98. A suite of modules and tools that simplify the creation of 
speech applications. (with Scott Hansma.) 


193. Scheduler, 1997 A dialog system for interacting with a personal calendar. Used to develop 
ideas in stack-based dialog control. (with Paul Constantinides and Scott Hansma). 


194. Cruiser, 1996 [a dialog system for access to auto registration information. Developed in 
conjunction with CGI, tested with the Pittsburgh Police Department. ] 


195. SpeechWear, 1995 [a PC-based wearable speech system, incorporating a speech hypertext 
browser. ] 


196. OM (Office Manager), 1990 [An environment supporting speech interaction with basic 
office applications, also an early example of a distributed multi-application client-server 
speech system. System was used for interface experiments. ] 


197. Voice Spreadsheet, 1989 [A spreadsheet system incorporating the Sphinx recognition 
system, used to conduct experiments on speech interfaces. ] 


i. Video Productions 


198. Carnegie Mellon Team Talk (2009) TeamTalk is a platform for multi-participant dialog 
between humans and robots. 


199. Communicator (March 2002) A telephone-based spoken dialog system for air travel 
reservations. 


200. Symphony (July 2001) A voice-enabled mobile maintenance system. 


201. Information access through speech (1997) An early example of a mobile, browser-based 
system. It was the first system to incorporate ASR into the browser (Mosaic) and to 
implement html tags for speech. 


202. The SpeechWear system (June 1995) 


203. Spoken Language Interfaces: The OM System (1990) The CMU Office Manager, a multi- 
application speech interface. [refereed, appears in CHI'91 video program] 


204. Examples of Voice Interfaces (1989) A voice spreadsheet and an early version of the OM 
PID system 
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205. Early CMU Speech Systems (1988) Voice Calculator and Voice PID 
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VII. INFORMATION REFLECTING EXTERNAL REPUTATION 


Invited Lectures: 

Invited Talk, SLT, Goa, India, December 2008 

Keynote Address, Catalog’04 (SemDial), July 2004. 

Speech Technology Status, Trends and Best Practices, Speech Tech, March 2004 
Invited talk, Boeing, 14 February 2003 

Invited talk, Microsoft Research, 13 February 2003 

Invited talk, Microsoft Research, 26 March 2001 

Keynote Address, SIGdial Workshop, 2 September 2001 

Invited talk, AT&T Research, 14 November 2000 

Invited presentations at ASAT, 1997, 1998 

"Speech Recognition Technology", AVIOS, 11 September 1995 

"Current research in speech recognition", ASAT, 14 June 1994 

"The spoken language interface", IBM Thornwood, 23 July 1990 

"Spoken language systems” National Communications Forum, 3 October 1989 
"Spoken language systems” Chicago Human Factors Society, 4 October 1989 
"Building interactive speech systems", National Technical University, 19 April 1989 


Workshops: 

NSF Panel on Spoken Dialog Corpus Composition and Annotation for Research, 2007 (co-chair) 
Speech in Mobile and Pervasive Environments (SiMPE), 2006-2009 (co-chair) 

Interspeech 2006 (organizing committee) 

SIGdial Workshop, Sapporo, Japan, July 5-7, 2003 (co-chair) 

NSF Workshop on Research Directions in Dialogue Processing, Edmonton, Canada, May 31 — 
June 1, 2003 (co-chair). 


VIII. PROFESSIONAL ACTIVITIES outside the University 


Organizational Activities: 
Board of Directors, AVIOS, 1999- 
Board of Directors, SIGdial, 2001- 


Committees: 
- DARPA Speech and Natural Language Program 
o Coordinating [steering] Committee, February 1991 - 1995 
- DARPA Speech and Natural Language Program 
o Multi-site ATIS Data Collection Working Group (MADCOW), Fall 1990 - 1995 
- ARPA Speech Technology Program 
o Broadcast Business News (H4) Evaluation Committee, 1995-1996. (chair) 
o ARPA Speech Recognition Workshop, 1996 (program committee) 
o Advisory Committee, DARPA Communicator Program, 1999-2002; chair, 2000-2002. 


Review Panels: 
- NSF Human-Computer Interactions Review Panel, Fall 2007 
- NSF Interactive Systems Group Review Panel, Spring 1995, 1997. 
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Program Committees 


IEEE Spoken Language Technology Workshop (Goa), 2008 (program) 

IEEE Spoken Language Technology Workshop (Aruba), 2006 (program) 

Tutorial and Research Workshop, Perception and Interactive Technologies (2006) (program) 
AAAI Workshop on Statistical and Empirical Approaches for Spoken Dialogue Systems, 
July 2006 (program) 

AAAI Workshop on Spoken Language Understanding, July 2005 (program) 

Automatic Speech Recognition and Understanding Workshop (Puerto Rico), 2005 (program) 
FLAIRS (Miami), May 2004. (program) 

Tutorial and Research Workshop on Affective Dialogue Systems (2004) (program) 

SIGdial 2003, 2004, 2006, 2007 (program) 

International Conference on Multimodal Interfaces (ICMI), 2006 (program) 

International Conference on Multimodal Interfaces (ICMI-PUI'03), 2003 (program) 

ISCA Tutorial and Research Workshop on Error Handling in Spoken Dialogue Systems. 
August 28-31, 2003 (program) 

HLT/EMNLP 2005 (program) 

HLT/NAACL 2003, 2004, 2006 (program) 

Workshop on Natural, Intelligent and Effective Interaction in Multimodal Dialogue Systems, 
2002 (program) 

Autonomous Agents & Multiagent Systems (Bologna), July 2002 (program) 

North American Association for Computational Linguistics (NAACL) 2000 (program) 


Editor 


ETRI (Electronic and Telecommunications Research Institute) Journal, 2006- 


Ad Hoc Reviewing [recent] 


CHI, 2006. 

ICSLP/Interspeech, 2004-2009. 

ICASSP, 2004. 

NAACL/HLT 2004, 2008. 

SIGdial, 2004-2007. 

ADS’04 Affective Dialogue Systems, 2004. 

Eurospeech, 2003. 

Innovation and Technology Commission, Hong Kong Special Administrative Region, 2002. 
ICSLP, 2002 


Professional memberships [current] 


AVIOS, Applied Voice Input/Output Society 

IEEE, Signal Processing Society 

ISCA — International Speech Communication Association 
ACL — Association for Computational Linguistics 

SIGdial — Special Interest Group in Dialogue (ACL/ISCA) 


Other activities: 


Between September 1996 and June 1999 I shared my time between Carnegie Mellon and the 
Microelectronics and Computer Technology Corporation (MCC) in Austin, Texas, where I 
was the Technical Director of the Intuitive Interfaces to Information Systems (13S) project. 
The I3S project was focused on the development of a discourse-based technology for dialog 
control. Recently the scope of the project has been expanded to the area of language-based 
personal information agents and the area of ubiquitous (language-based) device interfaces. 
Consultant in speech technologies. 


Advisory Board 
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- Cepstral, LLC (Pittsburgh, PA). 
XIII. Evidence of Teaching Performance 


15-800(a) “Speech Understanding by Machine” Spring 1994. 

- Designed a curriculum for a graduate course on speech understanding 

- Developed a sequence of 4 labs to acquaint students with speech technologies (speech 
characteristics, acoustic modeling, language modeling, and application design). 


11-754 “Dialog Systems Laboratory”, Spring, annually, 1998-2010 (ongoing). 

- Developed a laboratory course centered on hands-on development of speech applications 
embedded in a lecture and discussion format. Students learn to build working systems, either 
individually or in teams. 


11-716 “Seminar in Dialog Processing”. Fall, annually 1999-2010 (ongoing). 
- Developed a lecture and readings-based graduate course on automatic dialog processing, 
surveying major ideas in the field. 


XV. STUDENT SUPERVISION 


Current PhD Students: 

- Satanjeev Banerjee [primary advisor] 

- David Huggins-Daines [primary advisor] 
- Mohit Kumar [primary advisor] 

- Matthew Marge [primary advisor] 

- Long Qin [primary advisor] 


Current Masters Students: 
- Aasish Pappu [primary advisor] 


Graduated PhD Students: 

- Ananlada Chotimongkol, “Learning the Structure of Task-Oriented Conversations from the 
Corpus of In-Domain Dialogs”, Language Technologies Institute, May, 2008 [primary 
advisor] 

- Rong Zhang, “Making an Effective Use of Speech Data for Acoustic Modeling”, Language 
Technologies Institute, December, 2007 [primary advisor] 

- Dan Bohus, Computer Science Department, May 2007 “Error Awareness and Recovery in 
Task-Oriented Spoken Dialog Systems” [primary advisor] 


- Jahanzeb Sherwani, “Speech Interfaces for Information Access by Low-Literate Users in the 
Developing World” [co-advisor] 

- Ronald Rosenfeld, School of Computer Science, April, 1994. "Adaptive statistical language 
modeling: A maximum entropy approach" [co-advisor] 

- Alexander G. Hauptmann, School of Computer Science, July 1991. "Meaning from structure 
in natural language processing" [co-advisor] 


- Brian Langner, Language Technologies Institute, 2009. [Committee member] 
- Gregory Aist, Language Technologies Institute, 2002. [Committee member] 
- Stephanie Tomko, Language Technologies Institute, 2006. [Committee member] 


- Steven L. Teal, Graduate School of Industrial Administration, was expected Fall 1995 (but 
LOA). "A performance model of system delay and user strategy" [primary advisor] 
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Graduated Masters’ Students: 

- Aasish Pappu [expected 2010] 
- Dipanjan Das [2008] 

- Mohit Kumar [2007] 

- David Huggins-Daines [2006] 

- Satanjeev Banerjee [2005] 

- Nikesh Garera [2005] 

- Ananlada Chotimongkol [2001] 
- Alice Oh [2000] 

- Steven Reed [1995] 


External: 

- Opponent in PhD defense, Royal Institute of Technology (KTH), Sweden, 2007 
- Opponent in PhD defense, Linköping University, Sweden, 2006 

- External Reader for PhD, Tampere University, Finland, 2003. 


XVI. SERVICE AND COMMITTEE WORK 


- Language Technologies Institute Student Research Symposium, Fall 2003. (faculty co- 
advisor) 

- Language Technologies Institute Faculty Retreat, Fall 2001. (chair) 

- School of Computer Science Facilities Committee, 2002- 

- Carnegie Mellon University Faculty Senate, 2001-2002. (CSD representative) 
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